There is a wide variety of applications in which it is necessary or useful to perform a series of actions on or affecting a setup or system S in order to acquire knowledge about the behaviour of the system S. The setup S can be a natural system or an artificial (man-made) system—use of the expression “setup” does not imply that the system has been actively organized or configured (although that could be the case). Often the desired “knowledge” consists in learning some relationship that holds between a set of the system's parameters. In general, it is desirable to minimize the number of actions that must be carried out.
In general, any setup or system S can be described in terms of its own current state and the state of an environmental context in which the system finds itself. The system's current state can be described in terms of values taken by a number of different parameters characterizing the system (“system-state parameters”). In a similar way, the system's environmental context can be described in terms of values taken by a number of different parameters characterizing that context (“context parameters”). In many cases it is not possible to determine clearly whether a given parameter is a system-state or context parameter. Indeed this distinction is immaterial as far as the operation of the present invention is concerned. According both kinds of parameter will be referred to herein as system-state/context parameters.
Actions can be taken which affect the system S. For example, it may be possible to control a parameter which affects the system S, so that this parameter takes a selected value. The controlled (or “tuned”) parameter may be a system-state/context parameter but this is not obligatory. For a particular system S it is possible to select a set of parameters (or “action variables”) whose values will be tuned in actions that are intended to elicit information about the behaviour of the system S.
The present invention concerns the selection of actions, affecting a system S, which are to be taken deliberately with a view to observing how the system reacts or behaves. In the present document, the expression “action” is not intended to imply a necessity for physical motion or the operation of an actuator; on the contrary, the expression “action” relates to the deliberate setting of a set of one or more action variables to respective particular values. In many cases the tuned variables will relate to a physical quantity (e.g. the amount of water to supply to a crop, the level of a voltage to apply to an electrode, etc.), but this is not a requirement of the present invention.
It is helpful to explain what is meant in the present document by the expression “system-state/context/action space”.
Consider an application in which a user wishes to discover the relationships inherent in a setup S in which circuit boards are being photographed for quality control purposes. The circuit boards can have substrates made of different materials and, hence, can have different brightness values. The user wishes to discover how the exposure time and ambient illumination affect the contrast between the circuit boards and the conductive paths they carry, as observed in the photographic images. More particularly, he wishes to find out what conditions produce high-contrast images.
It is assumed that the photographic equipment includes a digital camera mounted over a conveyor which carries the circuit boards into a fixed position relative to the camera. An image-processing unit determines automatically the contrast between the circuit boards and the conductive traces they carry, by processing the image data generated by the digital camera. For simplicity, it is assumed that the image processing unit can accurately determine what within the image corresponds to the circuit board and what corresponds to the conductive traces.
This particular system S can be described in terms of the brightness of the circuit board substrate, BR, the ambient illumination, AI, and the exposure period, EP, at the time when the photographic images are generated. Only the exposure-period parameter EP can be set to different values under the user's control (or automatically), the other two parameters are considered to be outside the user's control. Thus the circuit board brightness, BR, and ambient illumination, AI, can be considered to be system-state/context parameters describing the system, and the exposure period EP can be considered to be an action variable whose value can be varied or “tuned”.
It can be considered that these parameters AI, BR and EP define a multi-dimensional space which, in this simple case, is a three-dimensional space as illustrated in FIG. 1. In a more realistic example the multi-dimensional space is likely to have considerably more dimensions than three, however this would be difficult, if not impossible, to represent in a drawing.
The above-described multi-dimensional space defined by the system-state/context parameter(s) and action variable(s) of a system is referred to in the present document as “system-state/context/action space”.
At any given moment, the above-mentioned example system can be described in terms of a vector defining the values taken by the circuit board brightness, BR, ambient illumination, AI, and the exposure duration, EP. FIG. 1 shows an example of a vector, A, corresponding to one particular combination of values of BR, AI and EP.
The cuboid shown in FIG. 1 corresponds to the system-state/context/action space for the contrast-measurement system S described above. It will be noted that FIG. 1 shows limits of 0 and 1 on the values of each system-state/context parameter and action variable. This represents the system-state/context/action space in the case where each of the system-state/context parameter values and action variable values is normalised so as to range from 0.0 to 1.0.